Conventionally, a smart audio system continuously listens to audio in a household for a wake word (also known as a trigger word). When the smart audio system detects a preset wake word, the smart audio system generates a request to a corresponding remote server system. The smart audio system sends audio data produced by a microphone of the smart audio system to the remote server system for processing. However, the smart audio system can receive sensitive information that was not intended by the user to be sent to the remote server system. For example, the user may have spoken a word that sounds like the wake word. Sensitive information or data sent to the remote server system via a computer network can also be subject to eavesdropping by third parties.